Papers with consistent tokenization
jp-evalb: Robust Alignment-based PARSEVAL Measures (2024.naacl-demo)
Copied to clipboard
| Challenge: | evalb is used for constituency parsing evaluation, but imposes constraints and requires consistent tokenization and sentence boundary outcomes. |
| Approach: | They propose an evaluation system designed to compute PARSEVAL measures, offering a viable alternative to evalb commonly used for constituency parsing evaluation. |
| Outcome: | The proposed evaluation system is based on an alignment method that aligns sentences and words when discrepancies arise. |
Tokenization Consistency Matters for Generative Models on Extractive NLP Tasks (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained sequence-to-sequence (seq2sequ) models have been widely used to solve extractive tasks, where parts of the input are extracted to form the desired output. |
| Approach: | They propose a simple fix to tokenization inconsistency that damages extractive nature of generative models by causing performance drop and hallucination. |
| Outcome: | The proposed model performs better in both in-domain and out-of-domain datasets with a notable average of +1.7 F1 gain when a BART model is trained on SQuAD and evaluated on 8 QA datasets. |